Reducing Queuing Stalls Caused by Data Prefetching

Data prefetching is an effective way to hide memory latency. Current microprocessors allow multiple prefetch requests to overlap and provide a queue structure to track the outstanding requests. A prefetch request can either wait or be dropped when there is no available room in the queue. Recognizing the effectiveness and accuracy of the prefetches generated by the compiler, recent architectures allow the selection of stalling or dropping to be visible to the compiler. Although such designs prove to be beneficial, they also expose a new kind of stall, queue full stall. Large percentage of queue full stall as high as 50% can be observed for SPEC CFU2000 on both UltraSparc IV+ and Itanium 2. In this paper, we explain the reasons for queue full stall. We propose three different approaches (prefetch scheduling, selectively dropping prefetches and using nontemporal locality hints) for reducing queue full stall and show the performance potential for both SPEC CPU2000 and SPEC CPU2006 [10]. By implementing the third approach in a dynamic binary optimization framework on Itanium 2, we are able to improve the performance of a set of kernel and SPEC CFP2000 programs by as much as 16.5%.